DEVICES, METHODS AND PROGRAM PRODUCTS FOR 
LOCATION AND ENVIRONMENTAL STAMPING OF IMAGES, 
COMMUNICATIONS AND OTHER APPLICATIONS 

5 Inventors: Edith H. STERN, Barry N. WILLNER 

BACKGROUND OF INVENTION 

10 1. Field of Invention : 

This invention relates to recording systems and methods of operation. More particularly, 
the invention relates to devices, methods and program products for location and environmental 
stamping of images, communications and other applications. 
^ 15 2. Description of Prior Art: 

'ti Location and environmental stamping of images, communications and other applications 

pi 

provide a record of importance similar to time stamping in message processing, time recording, 
m library activities and other applications. The availability of the omnipresent Global Positioning 
s Systems (GPS) and other locating systems facilitates location stamping of objects of interest, 

S 20 e.g.momuments, scenic sites, travel points or any view of interest related to the GPS coordinates 
^ in images or pictures, email and other communications. Further, the miniaturization of electronic 
™ devices for environmental measuring, e.g., temperature, pressure, humidity facilitates 

environmental recording. Location and environmental stamping add another dimension to 

recording objects of interest, particularly in imaging applications. The stampings are further 
25 enhanced by including in the recording descriptive text of the object of interest. Moreover, the 

GPS coordinates enable an image of an object interest to be stored in a network for later recall by 

a terminal. 

What is needed in the art is a pervasive device, system and method exploiting network 
and data processing techniques for location and environmental stamping of objects of interest 
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including descriptive text to enhance and readily recall recorded images for the general public, 
educational and business communities. 

Prior art related to recording objects of interest includes the following: 

In USP 5,506,644 entitled "Camera" issued April 9, 1996, discloses a position 
information recorder incorporated in a camera. A position-measuring unit receives position 
information via a GPS satellite and performs position measurements to calculate a position. A 
receiving state determination unit determines the state of receiving the position information. A 
memory stores the position measurement data, and a magnetic recording unit records data in the 
magnetic recording area of film. When position measurement data cannot be fetched, position 
measurement information stored in the memory immediately before is read out and recorded. 

USP 5,642,285 entitled "Outdoor Movie Camera GPS-Position and Time Code Data- 
Logging for Special Effects Production", issued June 24, 1997 discloses a GPS navigator 
receiver with a data output port for communicating a position estimate frame-by-frame and time 
codes in SMPTE format. The conventional date and time sequence output of the GPS navigation 
receiver is combined with a highly accurate one pulse per second (± 1 microsecond) to perform a 
timekeeping base for the time code output. A data logger records the position estimate with their 
corresponding time in time code format frame-by-frame in a database. Recorded position 
estimates are then used in special-effects post-production to direct the special effects animation, 
modeling kinetics, etc. in a composite, final film or video. 

USP 5,267,042 entitled "Image Pickup Device for Automatically Recording the Location 
Where an Image is Recorded", issued November 30, 1993, discloses an image pickup device 
body for recording an image object. A position detection means detects the present location of 
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the image pickup device body to record the resulting position data at the same time when the 
object image is recorded onto the image recording medium by the recording means. 

USP 5,515,042, entitled "Traffic Enforcement Device", issued May 7, 1996 discloses a 
traffic monitoring an evidence gathering device which can be mounted in a movable monitoring 
5 vehicle to monitor vehicle traffic speed. When a speeding violation is detected, the device 
generates a vehicle-identifying image having date, time, and vehicle speed and location 
information imposed thereon. The device includes a speed detector operable coupled with a 
camera and a global positioning monitor. The monitor is also connected to the camera. When a 
monitored vehicle's speeds exceeds a predetermined magnitude, the detector generates an output 
O 10 serving to initiate operation on the camera and satellite monitor and to deliver speed information 
z!^ to the camera. The camera creates a vehicle-identifying image and imposes on the image the 
speed and position information. 

01 

^ USP 5,910,986, entitled "Methods and Apparatus for Sending Electronic Signals", issued 

B 

□ June 8, 1999, discloses a time stamp and a location stamp included in a request for sensitive 
ry 15 information required to be sent over an open network, such as the hitemet. The time stamp 
y carries the time in which a previous page in the application was sent to the client. The location 

stamp carries the client address as provided for that previous page. Preferably, the stamps are 

encrypted and are thus protected at the xiser location. 

None of the prior art discloses a pervasive device and method of operation exploiting 
20 network and data processing techniques for correlating location and environmentally sensed 

information of an object of interest with stored information in a database in the device or in a 

network linked to the device for recording or stamping and readily recalling an object of interest 

in an image, communication or other medium. 
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SUMMARY OF THE INVENTION 

An object of the invention is a recording system and method of operation for sensing and 
incorporating in a medium, location and/or environmental sensed information related to an object 
of interest. 

Another object is a device and method of operation for recording in a medixim, location 
and/or environmental information, including descriptive text, related to an object of interest. 

Another object is a device and method of operation exploiting data processing techniques 
for supplementing sensed location and/or environmental information with additional information 
related to an object of interest for recording in a medium. 

Another object is a portable device linked to a network for collecting and correlating 
location and/or environmental information related to an object of mterest for recording in a 
mediimi. 

Another object is a portable device including software for imaging an object of interest 
and collecting, storing and supplementing location and/or environmental information related to 
the object for incorporation in a film or video or like media. 

Another object is a system and method for recalling and incorporating stored images in a 
communication according to location coordinates. 

These and other objects, features and advantages are achieved in a first embodiment in 
which a portable imaging device includes a data processing system, imaging and sensing devices 
for recording objects of interest and a terminal connection for transmitting the image to a 
network. The sensing devices record location and environmental conditions related to the object 
of interest. The device is linked to a positioning determining system, e.g. the Global Positioning 
System or other positioning system. The device records an image and environmental 
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information of an object of interest in a medium and incorporates into the image stored 
descriptive information of the object based on GPS or other location coordinates. The medium 
may be translated into an analog or digital picture incorporating the recorded location, 
environmental and descriptive information. In another embodiment, the device may be linked by 
wire or wirelessly via the terminal connection to a network including a server and a PC terminal 
or workstation. The recorded image, location and environmental information are transmitted by 
the device over the network to the server using a wireless connection. The server correlates 
stored descriptive information with the location and/or environmental sensed information for 
download to the PC terminal when the user accesses the server for display and/or editing of the 
image. At the user's direction, the PC prints out the image as a picture including the recorded 
location, environmental and descriptive information on the front, back or near the picture. In still 
another embodiment, a user visiting an object of interest sends a communication, e.g. an email 
over the network using a portable terminal. The email includes the GPS coordinates for the 
object as provided by location determining means incorporated in the portable terminal. The 
server includes a database of GPS coordinates and corresponding "thumbnail images" of objects 
of interest and incorporates into the email the "thxmibnail" image corresponding to the GPS 
coordinates. 

DESCRIPTION OF THE DRAWINGS 

The invention will be further xmderstood from the following detailed description of a 
preferred embodiment, taken in conjunction with an appended drawing, in which: 

Figure 1 is a representation of a system incorporating the principles of the present 
invention for collecting, storing and correlating image data with location and/or environmental 



YOR-92000-0301 
17123 1 



5 



1963-7393 




information relating to an object of interest for recall and/or incorporation in a medium, as a 
display or in a message or the like. 

Figure 2 is a block diagram of the portable-imaging device included in the system of 
Figure 1. 

5 Figure 3 is a flow diagram for processing image, location and environmental information 

in the portable imaging device of Figure 2. 

Figure 4 is a flow diagram using the system of Figure 1 for incorporating location and an 
environmental information in an image of an object of interest and/or recall of the image in a 
communication according to the location information 
C 10 DETAILED DESCRIPTION OF PREFERRED EMBODIMENT 

^ Figure 1 is an overview of the device and system for location and/or environmental 

y ^ 
m 

1=; stamping related to an image or commimication of object of interest. A user with a device 20, 
^ built in accord with the invention, captures an image of a local site 22 of interest. The device 20 
□ determines its location by means of accessing the GPS satellite system. One such satellite (50) is 
ru 1 5 shown in Figure 1 . The location coordinates are retained in a database (not shown contained in a 
^ memory 160 (See Fig. 2) along with the picture that was taken of the site, as one example of an 
object of interest. An environmental probe sensor 220 (See Figure 2), such as those available 
from TPI Corporation, 3 Greta Road, Grafton, MA, 01519, USA senses environmental 
conditions, e.g. temperature, pressure, humidity, etc. related to the picture and the location 
20 coordinates. Later, the picture and associated location coordinates and/or sensed environmental 
conditions are transferred to a PC device or workstation 30 in a digital commimication via a 
network 32, typically the Internet, and a server 40, as will be explained in more detail 
heremafter. The server 40 recognizes the coordinates of the site from a stored table and 
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generates a textual description of the site for download to the PC along wdth the picture, location 
and environmental condition associated with the picture. When the picture 60 is printed out by 
the device 30, the location, textual descriptions and/or environmental conditions are 
superimposed upon the picture. Altemately, the location could be printed near the picture or on 
the back. 

In an altemate embodiment, the server 40 may retum the textual description to the device 
20 which stores the textual description in the database for display of the image 60 in the 
viewfinder along with the textual description and environmental conditions. 



^^^Figure 2, the portable device 20 may be a digital camera which focuses an image 
through a COTventional optical system 1 10 onto a charged-coupled device (CCD) 170. A digital 
processing systenvl 80 in the camera is programmed to receive the output of the charge couple 
device, compress the omital image, and deliver the compressed image over bus 190 for storage in 
a memory 160 using well Know compression techniques. An antenna 130 couples a GPS unit 
140 to the satellite 50 (See FigNl) and receives location signals for the device 20. The unit 140 
processes the signals using calculations well known in the art to determine location coordinates 
of the image for storage in the memory\60 coupled to the bus 190. The environmental probe 
220 captures the environmental conditions ih digital form at the location and supply them to the 
data processing system 1 80 via the bus 1 90 for determining the environmental conditions, e.g. 
temperature, pressure, humidity using software applications provided by the probe manufacturer. 
A location mode selector switch 205 can be set to indicate the user's desire as to whether 
location and/or environmental information should be associated with the image. Camera control 



logic 150 described for example in USP 5,015,107 directs the data processing system to capture 
the current image from the CCD and directs both the location and environmental control logic to 
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capture the current location and environmental sensed information for storage in the memory 
leoNrhe camera control logic correlates the location coordinates and environmental information 
with the ccmtents of the image. The contents of the memory are commimicated through an 
external comnhmication link 208 to the network 32. One example of a network link is a 
5 Bluetooth WirelessK^onnection, developed by the Bluetooth Special Interest Group and 

described in the Bluetooth Protocol Architecture, Version 1, Document No. l.C. 120/1.0, dated 
August 25, 1999 and available from Nokia Corporation, (address to be supplied). A trigger 
switch 210 initiates operation ortjie device 20. 

In operation, when a user depresses the trigger button 210 to capture a picture or object of 
□ 10 interest, the digital processing system 180 is activated to take the output of the CCD in digital 
^- format, compress and output the image over the intemal bus 190 to the memory 160. If the 
z2 location mode selector switch 205 is operated, the location information associated with the 
> picture is obtained through the GPS interface and processing logic 140. The location coordinates 
P from the GPS interface are outputted over the intemal bus 190 and stored in the memory 160 

nj 1 5 with the digital image. The environmental information is collected by the sensor 220 and 

H= 

B transmitted to the memory over the intemal bus. In one embodiment, a connection is established 
by the unit 208 from the digital camera to a PC or workstation having Internet connectivity. 
Once the PC connectivity between the workstation and the portable device is established, the PC 
contains software, which prompts the user to select an editing mode. If the user wishes to 
20 actively edit the picture, an edit mode indicator is set. The image and the associated location and 
environmental information for the image are communicated to the PC. The PC signs on to a 
server to obtain descriptive material related to the location coordinates. The server based on the 
location coordinates accesses the server memory for a textual description related to the 
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coordinates and stored in advance in the server memory. In some instances, the user can select 
among several stored descriptions and the selected description is associated with the picture 
along with the location coordinates and environmental information. The picture is printed with 
the selected description, location coordinates and environmental information. If the Internet 
5 mode is not selected, a default description stored in a database in the camera is associated with 
the picture. The picture is then printed with the default description, location coordinates and 
environmental information in a manner well known in the digital cameras art. 
. ^^^"^igure 3 taken in conjunction with Figure 1 and 2 describe a process 300 inside the 
camera 20 Vhen an image 22 is recorded. The process is started in step 301 when a user presses 
3 10 the trigger butW 210 in step 303 to capture the image 22. The digital processing system 180 is 
lJ activated to take me output of the CCD 170 in step 305 and process the image in digital form in a 

manner well known i^the art, compress, and output the image over the intemal bus 190 to the 
J memory 160. In step 30\ a test is performed to determine if the location mode selection switch 

1 205 is operative. Normally the switch has an "ON" or "OFF"state. Altematively, the switch 

3 \ 

J 1 5 may have several states for displaying the image alone or with location information or with 
3 environmental information or conibined location and environmental information. If operative in 
the combined location/environmentalWte, the location information is acquired from the GPS 
unit 140 in step 390 for association with t^e image in step 310. Environmental information 
sensed by the probe 220 is processed in the s^tem 180 and combined with the location 
20 associated for the image in step 312. The image^d the associated location and environmental 
data are stored in the memory 160 in step 314. 

Returning to the test 307, if the location mode selector is "OFF" the image is stored in the 
memory 160 without the location and environmental information in step 314, after which the 



YOR-92000-0301 
17123J 



1963-7393 



process accesses the Bluetooth Wireless Connection 316 to connect to the server 40, via the PC 
30 using the Bluetooth Protocol, previously described. In step 318, after connection to the 
server, the device 20 transmits the image data and stored location and environmental information 
in a wireless message to the server for storage. 



y^^^Fagu^e 4 taken in conjunction with Figure 1 and 2 describes a process 400 for processing 
the image i\ttie device 20 by an external device, e.g. a PC 30 having an Internet connection. 
The process is u^tiated in step 401, and in step 403 a network connection is established between 
the digital camera 2\ and the PC 30 having Internet connectivity. In a preferred embodiment, a 
vdreless connection is established between the camera 20 and the PC 30 via a Bluetooth 
protocol. Once the PC connebtivity to the camera is established, the PC software prompts the 
user to choose the desired edit nmde in step 405. If the user wishes to actively edit the pictures, 
an edit mode indicator is set in step 4^7 otherwise the step 407 is bypassed. In step 409, the 
associated location and/or environmentamiformation are communicated to the PC via the 
database 162 in the device 20.or from the server 40. In the case of the server, the user accesses 
the server for the stored unage, location coordiMtes etc using standard Intemet protocols. 
The location information is used in step 410 to acce^ descriptive materials from the server 40 
via the Intemet connection. If edit mode is chosen in st^ 412, all available descriptions are 
presented to the user in step 414 for user selection in step 4r6. The user can select one of these 
descriptions, and m step 41 8, the selected description is associausd with the image. In step 420 
the image is printed as the picture 60 with the selected description aWor environmental 
conditions at the location. If edit mode had not been chosen, a default de^ription is associated 
with the picture in step 422. In step 480, the picture is then printed with the drfault description 
in a manner well known in the art of digital cameras. Steps 401 through 422 are repeated for 
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eabl^Dicture by returning to the entry point in step 401 until the last picture is printed in step 424 
upon wmqh the process ends. 

As an alternative to printing, in step 420 (480), the image with the selected descriptive 
text can be stored. This storage can take place in the camera device 20, in the associated PC 30, 
or in a server connected to the Internet. 

As an alternative to capturing an image of an object of interest with location and 
environmental information, another embodiment in Fig. 1 enables a user visiting a location to 
obtain a representative image from a remote database, in lieu of taking a picture, and include the 
representative image in an email as an indication of the sender location. In this embodiment, a 
laptop or Personal Digital Assistant (PDA) 24 includes an ability to determine location 
coordinates from the GPS satellite system 50 or other location coordinate determining means. 
The laptop is linked by wire or wireless to the network 32. The server 40 stores images 
according to their GPS or other coordinates. In generating an email, a user of device 24 at a 
location, in lieu of taking a picture, transmits the GPS coordinates thereof to the server as an 
image request before, during or after composing the message. The server in processing the 
image request recognizes the GPS or other location coordinates and correlates the coordinates 
with the stored images. The server returns at least one image to the laptop for inclusion in an 
email. The user composes and transmits the email to the recipient(s) with the image associated 
with the location at which the email was created. 

Alternatively, an email may be composed by a user including the GPS coordinates for the 
location provided by the laptop. The email system or recipient's system interacts with the server 
40 to obtain and insert the image in the message corresponding to the GPS coordinates. This 
embodiment enables a conmiercial enterprise to advertise an attraction, e.g. hotel or theme park, 
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in lieu of postcards of the attraction, by supplying images of the attraction to the server for 
distribution to travel agencies and others on an agreed fee basis. The enterprise may also provide 
incentives to the users of the portable device 20 to employ a 'thumbnail" image of the attraction 
or enterprise stored in the server 40 within the user's email. 

Having described the invention in a preferred embodiment, various changes can be made 
without departing from the spirit and scope of the invention as defined in the appended claims, in 
which 
We claim: 
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